The article presents DRIFT (Dissatisfaction-Refined Iterative Preference Training), a novel approach to preference learning that utilizes abundant implicit user dissatisfaction signals from real-world applications like conversational AI and code generation. By focusing on these dissatisfaction signals and dynamically sampling positive feedback, DRIFT improves performance on various benchmarks, surpassing existing methods and preserving exploratory capabilities in model training.